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ABSTRACT 



An attempt was made to determine whether a 



ncn-reaponse bias exists in the annual follow-up surveys of graduates 
of teacher education programs when the surveys focus on more than 
employment. Subjects were 291 graduates of teacher certification 
programs at a major university in the southeastern United States from 
Fall 1986 through Summer 1987. In an initial survey concerning 
employment r 88.6% of the subjects responded. A second follow-up was 
conducted that requested more detailed information, including 
evaluation of the teacher preparation program and career plans. A 
total of 227 subjects (78%) responded. Sixty-four subjects did not 
respond, and most of these non-respondents could not be contacted. 
Data were also collected from analyses of subjects' college records. 
Chi-square tests comparing respondents and non-respondents on 
employment, major, status, date of graduation, location, and gender 
revealed no significant differences for these variables* 
Non-respondents tended to be older than respondents, but no 
significant differences were found for grade point average and scores 
on the American College Tests and the National Teacher Examiiiation. 
Since the survey had a 78% response rate, it was encouraging to know 
that respondents were representative of the entire sample. Two tables 
provide the results of chi-square and Mann-Whitney comparisons of the 
study data. (SLD) 
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Teacher Education Follow-up Surveys: 
Are the Respondents Representative of the Group? 

Whenever a mail survey fails to achieve a high response rate, there is 
some question about a possible bias due to nonresponse by part of the 
population. Would the survey results have been different if all of those sent 
questionnaires had responded? Are there some essential differences between 
those who responded and those who did *xot? Researchers have attempted to 
examine differences between early and late respondents as well as 
nonrespondents in an attempt to find the answer. It is assumed that late 
respondents mig^t well have been nonrespondents, had additional mailings or 
attempts to contact them not been undertaken, and thus they were potential to 
nonrespondents. Ellis, Endo and Armer (1970), however, concluded that 
comparing early and late respondents does not produce the same results as 
comparing respc idents with nonrespondents. 

Comparison of the respondents with the target population may give an 
indication of the representativeness of the respo'' ^ents . This is possible 
only if variables of interest can be determined for the nonrespondents (or for 
the population as a whole ^ . There is some evidence that respondents differ 
from nonrespondents in various areas of research in that they are better 
educated, have higher intelligence or achievement scores (Clausen & Ford, 
1947; Ellis, Endo & Armer, 1970; Gannon, Nothern & Carroll, 1971; Macek & 
Miles, 1975; Nielsen, Moos, & Lee, 1978; Reuss, 1943), have More interest or 
involvement in the topic of the research (Donald, 1960; Reuss, 1943)), have 
interest in the topic under study (Donald, 1960; Brown & Wilkins, 1978) are 
involved in or have loyalty to the sponsoring organization (Donald, 1960; 
Reuss, 1943), are rural residents (Ellis, Endo, & Armer, 1970; Reuss, 1943), 



are highly valued employees and females (Gannon, Nothern, & Carroll, 1971; 
Nielsen, Moos, & Lee, 1978), have job security and active community 
involvement (Hockstim & Athanasopoulos, 1970) . 

Follow-up surveys are commonly used to obtain information from college 
graduat^js about their employment status, career plans, and perceptions of 
their undergraduate programs. Such surveys are not necessarily noted for high 
response rates, however, and have been found to vary from 10% to 100% (Boser, 
1988) . Even when there is a fairly high response rate, the question of 
nonresponse bias may be unanswered. In a college employment study of alumni 
conducted over forty years ago, those not employed were found to be 
underrepresented (Shuttleworth, 1941) . The present study was undertaken to 
determine if there was a nonresponse bias in the annual follow-up surveys of 
graduates that focus on more than employment. 

Method 

Two hundred ninety-seven individuals completed initial teacher 
certification programs at a major university in the southeast from fall 
quarter 1986 through summer quarter of 1987. Two hundred thirty completed 
certification programs while earning BS/BA degrees; 67 were 
postbaccalaureates, including 18 who were part of a special year-long 
intensive, externally funded program. All 297 individuals were considered 
subjects for the study. Six individuals were ultimately deleted from the 
group when all attempts to contact them failed. In effect, they were 
unreachables rather than nonrespondents . 
Procpdnrf^a 

Information was gathered initially in the fall of 1987 through an 
employment follow-up survey. Subjects were asked through a mail survey to 
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supply their current address, occupation (if employed), and employer on an 
enclosed post-paid return postcard • Subjects who were not teaching also had 
the option of noting a need for assistance from the Career Placement and 
Planning Office. A second mailing, including a second return postcard, was 
sent approximateJy one month later. Telephone calls were placed to 
nonrespondents in the state as a final attempt to obtain the information. Two 
hundred sixty-three of the 297 graduates (88.6%) were contacted in this study. 

A second follow-up survey was initiated in March to obtain more detailed 
information from graduates including their evaluation of the teacher 
preparation program and their career plans. A preliminary letter was sent to 
each of the 297 graduates, followed by the questionnaire one week later. 
Three follow-up mailings were sent, the second containing another copy of the 
questionnaire. Deletion of the names of six of the graduates because of 
inability to reach them left a total of 291. A total of 227 graduates (78%) 
returned completed questionnaires for analysis. This left 64 nonrespondents 
who had apparently received the questionnaires but chosen not to respond. 

Telephone calls were placed to local nonrespondents in an attempt to 
obtain their responses to the eiant program evaluation items on the survey for 
comparison with responses of those returning the questionnaires. However, 
only five of the local nonrespondents were contacted after repeated telephone 
attempts. It was clear, however, that the telephones had not been 
disconnected, as would have been the case if the individuals had relocated and 
not received their mail. In some cases, there was no answer to rep^aated 
calls, and it was possible that someone else (not the graduate) was receiving 
calls at that number. Because of the small number of nonrespondents 
contacted, comparison of their responses to the evaluation questions were not 
compared with those of the survey participants. 
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Occupation and location of nonrespondents were available Ifrom 
information in the fall employment survey. Occupation was analyzed in two 
ways. The first analysis combined public and private school teachers as one 
group and included all others as a second group. The second analysis created 
a third group consisting of those who were employed in education but not as 
public or private school teachers. Gender, age, status (graduate or 
postbaccalaureate) graduation date for graduates, undergraduate grade point 
average of graduates, ACT scores, and percentile scores on the core battery of 
the National Teachers Examination (communication, general knowledge, and 
professional knowledge tests) were available from college records. 

Chi-square tests were used to compare respondents and nonrespondents on 
employment, date of graduation, location and gender. An independent t-test 
was used to compare the two groups on GPA. Mann-Whitney nonparametric tests 
were used for comparison of scores on the ACT and the NTE. The .01 level of 
significance was used because of the number of statistical tests that were 
conducted . 

Results 

There were no significant differences between respondents and 
nonrespondents on the chi-square comparisons for gender, location, graduation 
date of graduates, major, status, and occupation (see Table 1) . The t-test 
comparing respondents and nonrespondents (graduates only) on undergraduate 
cumulative grade point average was also not significant (t=0.88, p=.379) . The 
Mann-Whitney comparisons showed a significant difference in age (with 
nonrespondents older than respondents), but no significant differences on ACT 
or any of the three NTE core battery tests. 
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Discussion 

Nonrespondents were found to be similar to respondents on all variables 
except age, with the nonrespondents being older than the respondents. Since 
the survey had a 78% response rate, it is encouraging to know that the 
respondents are typical of the population under study and that a 78% response 
rate may provide a true representation of the group. The nonrespondents who 
were the focus of this study were all college graduates, some with 
postbaccalaureate experiences, and individuals who might be expected to have 
some loyalty to the sponsoring institution. Those factors have been 
associated with higher probability of participation in mail surveys . This 
study did not find a significant gender difference, as was found by previous 
researchers, however there was a trend in this direction with 80% of the 
females responding while only 71% of the males did so. 

One might expect that those teciching would respond at a higher rate to 
the institution that prepared them to teach than those not teaching. This was 
not the case, however. The response rate for those teaching in public or 
private schools was 81%. Those employed in other capacities in the field of 
education responded at the highest rate (92%), followed by those employed in 
other areas, in graduate school, or not employed (88%) . This is somewhat 
inconsistent with the results of Shuttlevorth (1941), who found those not 
employed to be underrepresented in an enployment survey. The context is 
similar: the survey wao undertaken to determine those employed after college. 
The present study is more specific, however, in that it focuses on a 
particular type of employment for which the institution prepared the 
individual. First-year teachers frequently find themselves spending much of 
their evenings and weekends preparing for their classes or grading papers. 
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Their employment demands more of their time than many other occupations, and 
it may be that those not teaching had more leisure time in which to respond. 

The major question still unanswered is whether noiirespondents differ 
from respondents in their evaluation of their teacher preparation programs. 
Attempts made to contact local nonrespondents by telephone to determine 
whether or not their evaluative ratings of the teacher preparation program 
were consistent with those of the respondents were not very successful. The 
five local nonrespondents who were ultimately reached after numerous telephone 
calls were insufficient to provide credible information for comparison with 
the large group of respondents. In talking with nonrespondents on the 
telephone, it was obvious that some were reluctant to evaluate their program, 
stating that they were not in a good position to makre judgments about the 
program since they had not taught. Some prompting was necessary before they 
would make even tentative ratings. Further study of this particular aspect of 
the nonresponse effect should be made in the future. 
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TABLE 1 



Chi -Square Comparisons of Respondents and Nonrespondents 



Respondents Nonrespondents 



X2 



Gender 

Male 
Female 

Location 

Local 
In state 
Out of state 



44 
183 



70 
124 
33 



Graduation date (graduates only) 



December 
March 
June 
August 

Major 



79 
17 
63 
12 



18 
45 



21 
32 
11 



25 
3 

18 
4 



1.960 



0.493 



0.842 



.162 



.781 



.839 



Art/Music 
Health/P.E. 
Special Ed. 
Tech. & Adult 
Elementary 
Secondary 

Status 



13 
8 
25 
20 
75 
86 



6 
5 
8 
6 

20 
19 



4.167 



.526 



Graduate (BS/BA) 172 

Postbaccalaureate 55 

Occupation 

Teaching 136 

Not teaching 91 

Teaching 136 

Employed in Ed, 33 

Other 58 



52 
12 



31 
11 

31 
3 
8 



0.565 



.452 



2.348 



3.162 



.126 



.206 



10 



1 



TABLE 2 

RESULTS OF MANN-MHITNEY COMPARISONS 







n 


u 


z 


pa 


Age 




283 


4799 


3.0167 


.0026 


ACT 




182 


2737 


.6779 


.4979 


NTE 


Communication 


228 


4141 


.7503 


.4531 


NTE 


Gen. Knowledge 


228 


4347 


.2501 


.8025 


NTE 


Prof. Knowledge 


228 


4423 


.0656 


.9477 



^ two-tailed probability 
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